Why do some probabilistic forecasts lack reliability?

In this work, we investigate the reliability of the probabilistic binary forecast. We mathematically prove that a necessary, but not sufficient, condition for achieving a reliable probabilistic forecast is maximizing the Peirce skill score (PSS) at the threshold probability of the climatological base rate. The condition is confirmed by using artificially synthesized forecast-outcome pair data and previously published probabilistic solar flare forecast models. The condition gives a partial answer as to why some probabilistic forecast system lack reliability, because the system, which does not satisfy the proved condition, can never be reliable. Therefore, the proved condition is very important for the developers of a probabilistic forecast system. The result implies that those who want to develop a reliable probabilistic forecast system must adjust or train the system so as to maximize PSS near the threshold probability of the climatological base rate.Comment: 12 pages, 6 figures, 1 table, accepted for publication in the Journal of Space Weather and Space Climate (JSWSC

Interplanetary particle transport simulation for warning system for aviation exposure to solar energetic particles

Solar energetic particles (SEPs) are one of the extreme space weather phenomena. A huge SEP event increases the radiation dose received by aircrews, who should be warned of such events as early as possible. We developed a warning system for aviation exposure to SEPs. This article describes one component of the system, which calculates the temporal evolution of the SEP intensity and the spectrum immediately outside the terrestrial magnetosphere. To achieve this, we performed numerical simulations of SEP transport in interplanetary space, in which interplanetary SEP transport is described by the focused transport equation. We developed a new simulation code to solve the equation using a set of stochastic differential equations. In the code, the focused transport equation is expressed in a magnetic field line coordinate system, which is a non-orthogonal curvilinear coordinate system. An inverse Gaussian distribution is employed as the injection profile of SEPs at an inner boundary located near the Sun. We applied the simulation to observed SEP events as a validation test. The results show that our simulation can closely reproduce observational data for the temporal evolution of particle intensity. By employing the code, we developed the WArning System for AVIation Exposure to Solar energetic particles (WASAVIES).Comment: 23 pages, 11 figures, accepted for publication in Earth, Planets and Spac

Verification of operational solar flare forecast: Case of Regional Warning Center Japan

In this article, we discuss a verification study of an operational solar flare forecast in the Regional Warning Center (RWC) Japan. The RWC Japan has been issuing four-categorical deterministic solar flare forecasts for a long time. In this forecast verification study, we used solar flare forecast data accumulated over 16 years (from 2000 to 2015). We compiled the forecast data together with solar flare data obtained with the Geostationary Operational Environmental Satellites (GOES). Using the compiled data sets, we estimated some conventional scalar verification measures with 95% confidence intervals. We also estimated a multi-categorical scalar verification measure. These scalar verification measures were compared with those obtained by the persistence method and recurrence method. As solar activity varied during the 16 years, we also applied verification analyses to four subsets of forecast-observation pair data with different solar activity levels. We cannot conclude definitely that there are significant performance difference between the forecasts of RWC Japan and the persistence method, although a slightly significant difference is found for some event definitions. We propose to use a scalar verification measure to assess the judgment skill of the operational solar flare forecast. Finally, we propose a verification strategy for deterministic operational solar flare forecasting.Comment: 29 pages, 7 figures and 6 tables. Accepted for publication in Journal of Space Weather and Space Climate (SWSC

Why do some probabilistic forecasts lack reliability?

In this work, we investigate the reliability of the probabilistic binary forecast. We mathematically prove that a necessary, but not sufficient, condition for achieving a reliable probabilistic forecast is maximizing the Peirce Skill Score (PSS) at the threshold probability of the climatological base rate. The condition is confirmed by using artificially synthesized forecast–outcome pair data and previously published probabilistic solar flare forecast models. The condition gives a partial answer as to why some probabilistic forecast system lack reliability, because the system, which does not satisfy the proved condition, can never be reliable. Therefore, the proved condition is very important for the developers of a probabilistic forecast system. The result implies that those who want to develop a reliable probabilistic forecast system must adjust or train the system so as to maximize PSS near the threshold probability of the climatological base rate

Development of a coronal mass ejection arrival time forecasting system using interplanetary scintillation observations

Coronal mass ejections (CMEs) cause disturbances in the environment of the Earth when they arrive at the Earth. However, the prediction of the arrival of CMEs still remains a challenge. We have developed an interplanetary scintillation (IPS) estimation system based on a global magnetohydrodynamic (MHD) simulation of the inner heliosphere to predict the arrival time of CMEs. In this system, the initial speed of a CME is roughly derived from white light coronagraph observations. Then, the propagation of the CME is calculated by a global MHD simulation. The IPS response is estimated by the three-dimensional density distribution of the inner heliosphere derived from the MHD simulation. The simulated IPS response is compared with the actual IPS observations made by the Institute for Space-Earth Environmental Research, Nagoya University, and shows good agreement with that observed. We demonstrated how the simulation system works using a halo CME event generated by a X9.3 flare observed on September 5, 2017. We find that the CME simulation that best estimates the IPS observation can more accurately predict the time of arrival of the CME at the Earth. These results suggest that the accuracy of the CME arrival time can be improved if our current MHD simulations include IPS data.Comment: 39 pages, 6 figures, accepted for publication in Earth, Planets and Spac

OCTAD-S: Digital Fast Fourier Transform Spectrometers by FPGA

We have developed a digital fast Fourier transform (FFT) spectrometer made of an analog-to-digital converter (ADC) and a field-programmable gate array (FPGA). The base instrument has independent ADC and FPGA modules, which allow us to implement different spectrometers in a relatively easy manner. Two types of spectrometers have been instrumented, one with 4.096 GS/s sampling speed and 2048 frequency channels and the other with 2.048 GS/s sampling speed and 32768 frequency channels. The signal processing in these spectrometers has no dead time and the accumulated spectra are recorded in external media every 8 ms. A direct sampling spectroscopy up to 8 GHz is achieved by a microwave track-and-hold circuit, which can reduce the analog receiver in front of the spectrometer. Highly stable spectroscopy with a wide dynamic range was demonstrated in a series of laboratory experiments and test observations of solar radio bursts.Comment: 20 pages, 7 figures, accepted for publication in Earth, Planets and Spac

Effect of Finite Larmor Radius on the Cosmic Ray Penetration into an Interplanetary Magnetic Flux Rope

We discuss a mechanism for cosmic ray penetration into an interplanetary magnetic flux rope, particularly the effect of the finite Larmor radius and magnetic field irregularities. First, we derive analytical solutions for cosmic ray behavior inside a magnetic flux rope, on the basis of the Newton-Lorentz equation of a particle, to investigate how cosmic rays penetrate magnetic flux ropes under an assumption of there being no scattering by small-scale magnetic field irregularities. Next, we perform a numerical simulation of a cosmic ray penetration into an interplanetary magnetic flux rope by adding small-scale magnetic field irregularities. This simulation shows that a cosmic ray density distribution is greatly different from that deduced from a guiding center approximation because of the effect of the finite Larmor radius and magnetic field irregularities for the case of a moderate to large Larmor radius compared to the flux rope radius.Comment: 17 pages, 14 figures, accepted for publication in The Astrophysical Journa

Nowcast and forecast of galactic cosmic ray (GCR) and solar energetic particle (SEP) fluxes in magnetosphere and ionosphere – Extension of WASAVIES to Earth orbit

Real-time estimation of cosmic-ray fluxes on satellite orbits is one of the greatest challenges in space weather research. Therefore, we develop a system for nowcasting and forecasting the galactic cosmic ray (GCR) and solar energetic particle (SEP) fluxes at any location in the magnetosphere and ionosphere during ground-level enhancement (GLE) events. It is an extended version of the WArning System for AVIation Exposure to SEP (WASAVIES), which can determine event profiles by using real-time data of the count rates of several neutron monitors (NMs) at the ground level and high-energy proton fluxes observed by Geostationary Operational Environmental Satellites (GOES) satellites. The extended version, called WASAVIES-EO, can calculate the GCR and SEP fluxes outside a satellite based on its two-line element (TLE) data. Moreover, organ absorbed-dose and dose-equivalent rates of astronauts in the International Space Station (ISS) can be estimated using the system, considering its shielding effect. The accuracy of WASAVIES-EO was validated based on the dose rates measured in ISS, as well as based on high-energy proton fluxes observed by POES satellites during large GLEs that have occurred in the 21st century. Agreement between the nowcast and forecast dose rates in ISS, especially in terms of their temporal structures, indicates the usefulness of the developed system for future mission operations